Runtime Data Flow Graph Scheduling of Matrix Computations with Multiple Hardware Accelerators

Abstract In our previous work, we have presented a systematic methodology for parallelizing dense matrix computations using a separation of concerns between the code that implements a linear algebra algorithm and a runtime system that exploits parallelism for which only relatively simple scheduling algorithms were used to parallelize a wide range of dense matrix computations. We have extended the runtime system to utilize multiple hardware accelerators yet constrained its use to a particular scheduling algorithm. In this paper, we develop a new domain-specific scheduling algorithm that addresses load balance and data locality simultaneously, which is motivated by queueing theory. In order to apply this domain-specific scheduling algorithm for utilizing multiple hardware accelerators, we implement a new software managed cache coherency mechanism that allows for the use of any scheduling algorithm. We provide performance results that validate that our domain-specific scheduling algorithm consistently attains exceptional performance on a homogeneous multicore platform and heterogeneous platform with multiple hardware accelerators.